Telegram Group Search
Django + Pgvector + LLMs = Semantic Search and AI Agent Powered Document Analytics

Hi, folks, sharing my latest open source Django project to experiment with Django-powered document analytics tools. I've worked on OpenContracts for a number of years now. While it started out as a tool to label and annotate documents, thanks to the recent advances in LLMs and vector databases, I've released a new version with a bunch of cool features to use LLMs, vector search and AI Agents. It keeps amazing me how Django keeps getting more and more capable with age!

I had to share!

Some Screen Captures:

You can upload documents and they're automatically parsed by layout and their vector embeddings are stored in Django via pgvector

Data extracted from documents is traceable back to the source in the document

Key Features:

1. Manage Documents - Manage document collections
2. Layout Parser - Automatically extracts layout features from PDFs
3. Automatic Vector Embeddings - generated for uploaded PDFs and extracted layout blocks
4. Pluggable microservice analyzer architecture - to let you analyze documents and automatically annotate them
5. Human Annotation Interface - to manually annotated documents, including multi-page annotations.
6. LlamaIndex Integration - Use our vector stores (powered by pgvector) and any manual or automatically annotated features to let an LLM intelligently answer questions.
7. Data Extract - ask multiple questions across hundreds of documents using complex

/r/django
https://redd.it/1dn1exe
Flask Mega Tutorial Migration problems

I'm following the 2023 Flask Mega Tutorial and I can't get the migration portion to work for me.

I don't want to use sqlite, so I have an AWS RDS for MySQL instance running and have confirmed that I can connect to it. I can make changes and run queries via MySQL Workbench. But no matter what I do I can't get flask migrate to see the models I'm defining and make the changes. The flask db init command works fine.

models.py

\_\_init\_\_.py

Anyone have any insight? I'm pulling my hair out trying to solve this.

/r/flask
https://redd.it/1dmzswg
Admin panel no longer synced with friend

I’m working on a project with a friend, and our admin panel would usually be synced. For example, if they made a superuser, then it would show up under Users on my admin panel and vice versa. However, I was working on the project today and realized our databases weren’t synced anymore, and it was causing issues in the project. Any general idea why this could happen and how to fix it?

/r/django
https://redd.it/1dn5gob
Need suggestions?

I am trying to build a forum for share marker investors in India as a part of my pet project.

Can you suggest the UI/UX that you find useful in this website.

I am very bad at front end and looking forward to learning React with tailwindcss to implement front end in near future.

One improvement i can think of is a typeahead in the search bar where users can see the possible matches as they type.

You can see the website here
https://rallyproduction.azurewebsites.net/

/r/django
https://redd.it/1dn8b97
Dynamically invalidate cache after database update

I've developed a Flask app where the frontend content is currently embedded directly in the HTML files. For SEO and easier content management, I want to move this content to a database so that I can update it via an Admin Panel.

Previously, I used Flask-Caching, but I encountered an issue where I had to set a timeout to see content updates. I want to improve this by automatically invalidating the cache when content changes. Here's my approach:

myapp/models.py

# myapp/models.py
from sqlalchemy import event
from myapp.extensions import db, cache

class PageSection(db.Model):
__tablename__ = 'page_section'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String)
content_html = db.Column(db.String)
created_at = db.Column(db.DateTime)
updated_at = db.Column(db.DateTime)

def invalidate_cache(mapper, connection, target):
cache.delete('page_section_cache_key')

event.listen(PageSection, 'after_update', invalidate_cache)
event.listen(PageSection, 'after_insert', invalidate_cache)
event.listen(PageSection, 'after_delete', invalidate_cache)



myapp/routes/pages.py

from myapp.models import invalidate_cache, PageSection

@app.route('/page/<section_id>')
@cache.cached(timeout=60*60, key_prefix=lambda: f'page_view_{section_id}')
def page_view(section_id):
page_section = PageSection.query.get(section_id)
return render_template('page.html', section=page_section)


My question is: Is this a good approach to ensure that the cached content is updated dynamically, and how can I improve it? Are there better ways to handle caching with dynamic content in Flask?



/r/flask
https://redd.it/1dmq9ol
Monday Daily Thread: Project ideas!

# Weekly Thread: Project Ideas 💡

Welcome to our weekly Project Ideas thread! Whether you're a newbie looking for a first project or an expert seeking a new challenge, this is the place for you.

## How it Works:

1. **Suggest a Project**: Comment your project idea—be it beginner-friendly or advanced.
2. **Build & Share**: If you complete a project, reply to the original comment, share your experience, and attach your source code.
3. **Explore**: Looking for ideas? Check out Al Sweigart's ["The Big Book of Small Python Projects"](https://www.amazon.com/Big-Book-Small-Python-Programming/dp/1718501242) for inspiration.

## Guidelines:

* Clearly state the difficulty level.
* Provide a brief description and, if possible, outline the tech stack.
* Feel free to link to tutorials or resources that might help.

# Example Submissions:

## Project Idea: Chatbot

**Difficulty**: Intermediate

**Tech Stack**: Python, NLP, Flask/FastAPI/Litestar

**Description**: Create a chatbot that can answer FAQs for a website.

**Resources**: [Building a Chatbot with Python](https://www.youtube.com/watch?v=a37BL0stIuM)

# Project Idea: Weather Dashboard

**Difficulty**: Beginner

**Tech Stack**: HTML, CSS, JavaScript, API

**Description**: Build a dashboard that displays real-time weather information using a weather API.

**Resources**: [Weather API Tutorial](https://www.youtube.com/watch?v=9P5MY_2i7K8)

## Project Idea: File Organizer

**Difficulty**: Beginner

**Tech Stack**: Python, File I/O

**Description**: Create a script that organizes files in a directory into sub-folders based on file type.

**Resources**: [Automate the Boring Stuff: Organizing Files](https://automatetheboringstuff.com/2e/chapter9/)

Let's help each other grow. Happy

/r/Python
https://redd.it/1dmzzup
BM25 for Python: Achieving high performance while simplifying dependencies with BM25S

Hello fellow Python enthusiasts :)

I wanted to share bm25s, a new lexical search library that fully implemented in Python (via numpy and scipy) and is quite fast.

Blog Post

GitHub Repository

Here is a comparison of BM25S and Elasticsearch in a single-threaded setting (calculated on popular datasets from the BEIR benchmark): https://bm25s.github.io/assets/comparison.png

It was designed to improve upon existing Python implementations, such as the widely used rank-bm25 by being significantly faster; all while being very straightforward to use in Python.

After installing with pip install bm25s, here's the code you'd need to get started:

import bm25s

# Create your corpus here
corpus =
"a cat is a feline and likes to purr",
"a dog is the human's best friend and loves to play",
"a bird is a beautiful animal that can fly",
"a fish is a creature that lives in water and swims",


# Create the BM25 model and index the corpus


/r/Python
https://redd.it/1dmwfbf
Can't get request.POST with HTMX + Checked Attribute

I'm printing the POST result like this:

print(self.request.POST)

In the template, I'm trying to POST the name and value of the input field. It works when the checked attribute is not there but gives me an empty QueryDict <QueryDict: {}> when it's there as per below.

<div class="form-check form-switch">
                 <input
                   class="form-check-input"
                   type="checkbox"
                   role="switch"
                   id="survey-{{ survey.pk }}"
                   name="button"
                   value="{{ survey.published }}"
                   hx-post="{% url 'surveys:survey-toggle-status' %}"
                   hx-trigger="click"
                   hx-target="#survey-table"
           

/r/django
https://redd.it/1dn85zr
Learn Restful API with Flask

🚀 Mastering RESTful APIs with Python and Flask 🚀

Are you ready to take your skills to the next level? I'm excited to announce the launch of my new Udemy course, designed to help you achieve your goals and excel in your field!🎓 Course Highlights:

* Comprehensive, easy-to-follow lessons
* Practical exercises and real-world examples
* Learn at your own pace, from anywhere

💡 Why Enroll?Whether you're a beginner looking to get started or a seasoned professional seeking to refine your skills, this course has something for everyone. Gain in-depth knowledge, practical skills, and the confidence to succeed.

🔗 Enroll Now: [https://www.udemy.com/course/mastering-restful-apis-a-comprehensive-guide/?referralCode=727DF5F0596A9494D3B5](https://www.udemy.com/course/mastering-restful-apis-a-comprehensive-guide/?referralCode=727DF5F0596A9494D3B5)

Join me on this exciting learning journey and unlock your full potential. See you in the course!

#Udemy #Python #flask #OnlineLearning #SkillDevelopment #Education #CourseLaunch #LearnFromHome

/r/flask
https://redd.it/1dnamd8
Flask Spotify API Authorization

Hey guys, i'm working on a flask project with spotify api and i'm kind of stuck on the authorization aspect. I already had everything built on flask but decided i wanted a frontend with react so i'm trying to send an access code from flask to react.

This is a nodejs, react tutorial i'm following to set up the authorization - [https://developer.spotify.com/documentation/web-playback-sdk/howtos/web-app-player\](https://developer.spotify.com/documentation/web-playback-sdk/howtos/web-app-player)

Everything works fine until here - Spotify code in nodejs:

request.post(authOptions, function(error, response, body) {
if (!error && response.statusCode === 200) {
var accesstoken = body.accesstoken;
res.redirect('/')
}
});
})

The code is stored locally then requested through an endpoint. I can't do the same with flask as the function will return and all variables and sessions reset. I'm currently sending the access code to frontend through the url. My code -

code = request.args.get('code')
response = util.tokenHeadersData(code)
if response:
return

/r/flask
https://redd.it/1dnkmos
Reladiff - High-performance diffing of large datasets across databases

Hi everyone!

I'm here to announce my open-source project Reladiff.

I hope some of you will find it useful!

### What My Project Does

Reladiff is a python library for diffing data across databases (e.g. postgres<->snowflake), and it can handle very large tables with blazing speeds, by running the diff in the database itself.

The API is pretty simple, and highly customizable. Here's the "Hello World":

from reladiff import connecttotable, difftables

table1 = connect
totable("postgresql:///", "tablename", "id")
table2 = connecttotable("mysql:///", "tablename", "id")

sign: Literal['+' | '-']
row: tuple[str, ...]
for sign, row in diff
tables(table1, table2):
print(sign, row)

### Target Audience

- Data professionals
- DevOps engineers
- System administrators.

Reladiff is safe for use in production.

### Comparison

Reladiff is a fork of a project called "data-diff". I was the main developer for data-diff until last year. It was recently abandoned and archived by its sponsoring company, which is why I'm doing this fork. I kept it mostly as-is, but I fixed the documentation, removed all the tracking code, and the dbt integration.

Other than that, I'm not aware of any relevant open-source alternative. But I'll be

/r/Python
https://redd.it/1dncta4
Naruto Hands Seals Detection (Python project)

I recently used Python to train an AI model to recognize Naruto Hands Seals. The code and model run on your computer and each time you do a hand seal in front of the webcam, it predicts what kind of seal you did and draw the result on the screen. If you want to see a detailed explanation and step-by-step tutorial on how I develop this project, you can watch it here.

/r/Python
https://redd.it/1dng3t3
D which universities and research centers are focusing on adversarial machine learning (especially in Germany) ?

Hey people, I'm curious about where core adversarial learning research is going on. Looking at the publications, I do see several papers from various organizations but rarely do I see an organization advertise that they are focusing on adversarial learning as a field. Especially so for the universities and institutes in Germany.

/r/MachineLearning
https://redd.it/1dnctew
Threading/concurrency and flask

I am trying to make a flask app that does some processing with an LLM in the background. My naive approach was to check the database every second for unprocessed entries and then process them one by one. I did this using Python's threading library. It seems this doesn't actually work when using a real server like gunicorn.

From what I have seen the proper way to do this is using Celery. Celery though seems to work using RabbitMQ, which I have no idea about. The examples seem to be built around redis, wheras my site is built around SQLAlchemy (currently with SQLite for testing but this could change). I don't really get why work queues are necessary. Is there a better way of doing this or do I need to bite the bullet and learn celery?

/r/flask
https://redd.it/1dnn2i5
Tuesday Daily Thread: Advanced questions

# Weekly Wednesday Thread: Advanced Questions 🐍

Dive deep into Python with our Advanced Questions thread! This space is reserved for questions about more advanced Python topics, frameworks, and best practices.

## How it Works:

1. **Ask Away**: Post your advanced Python questions here.
2. **Expert Insights**: Get answers from experienced developers.
3. **Resource Pool**: Share or discover tutorials, articles, and tips.

## Guidelines:

* This thread is for **advanced questions only**. Beginner questions are welcome in our [Daily Beginner Thread](#daily-beginner-thread-link) every Thursday.
* Questions that are not advanced may be removed and redirected to the appropriate thread.

## Recommended Resources:

* If you don't receive a response, consider exploring r/LearnPython or join the [Python Discord Server](https://discord.gg/python) for quicker assistance.

## Example Questions:

1. **How can you implement a custom memory allocator in Python?**
2. **What are the best practices for optimizing Cython code for heavy numerical computations?**
3. **How do you set up a multi-threaded architecture using Python's Global Interpreter Lock (GIL)?**
4. **Can you explain the intricacies of metaclasses and how they influence object-oriented design in Python?**
5. **How would you go about implementing a distributed task queue using Celery and RabbitMQ?**
6. **What are some advanced use-cases for Python's decorators?**
7. **How can you achieve real-time data streaming in Python with WebSockets?**
8. **What are the

/r/Python
https://redd.it/1dnsczh
tell me how I can implement the described feature In my code

I have a drop down menu with iPhones that are an option for repair. for value= or the back end so to say I figured I should include the price , how do I Make it so they get charged that amount at checkout. should I use stripe ?

/r/flask
https://redd.it/1dms9c4
Django website template to speed up your development

Hi everyone!

I saw couple of posts about how tedious and time consuming its to get django website from scratch, especially from freelancers.

So to make things easier I put together a base template, so you can build on that, you can check it out here: [https://github.com/PaulleDemon/Django-website-template](https://github.com/PaulleDemon/Django-website-template)

You can also check out the [Django template demo](https://django-website-template.vercel.app/).

**Some of the features of Django template include**

* **Production ready**: Comes with configurations for production, you can start deploying immediately to Render.com, vercel.com, railway.app etc.
* **Responsive UI**: Comes with a responsive base template, so you can focus on other parts of your websites.
* **Landing page**: Comes with a landing page you can modify.
* **Contact us page**: An inquiry form, to allow your users to contact.
* **Blog with Trix Editor:** An integrated Trix editor on admin panel to manage and write blogs.
* **Custom User model**: Comes with an existing custom user model you can modify.
* **Tailwind Css:** For rapid prototyping and development

You can check out all the feature [here](https://github.com/PaulleDemon/Django-website-template?tab=readme-ov-file#what-features-does-django-template-include).


**Couple of technical details:**

* It comes with Function based views.
* Uses tailwind CSS
* Supports Sqlite / Postgres Database
* You can check the readme.md, to know how to get started and deploy it to production.


**Who is

/r/django
https://redd.it/1dnhlf5
Automating telemetry capture using Python bytecode

This article covers my journey attempting to capture telemetry automatically using Python code. It ended up being super complex and too much for our company to maintain. I'm sharing it to get some insights to see if folks have done anything similar and have ideas!

https://jaywhy13.hashnode.dev/automated-telemetry-capture-via-python-bytecode-modification

/r/Python
https://redd.it/1dnu0po
2024/06/25 07:14:11
Back to Top
HTML Embed Code: